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Executive Summary 


Career Technical Education (CTE) is a priority of nearly every governor, and many state policymakers 
recognize that it is a critical strategy to expand access to opportunity and train the workforce of the 
future. But as CTE continues to gain recognition and priority within the states, state leaders must 
improve the quality and effectiveness of their CTE data to demonstrate impact, improve the quality 
of their programs, and ensure equitable access to high-quality opportunities. 


At this moment, states have a lot of room for improvement. Less than half of State CTE Directors say 
their CTE data systems provide the information they need to assist in making decisions about CTE 
program quality and initiatives at both the secondary and postsecondary levels. Having access to this 
information should be an urgent priority for states. With growing attention and investments in CTE, 
State Directors will face new pressure to be transparent and demonstrate impact. They — and other 
CTE stakeholders — must have confidence in their data to make informed decisions about CTE. 


fall 2018, Advance CTE conducted a survey of State Directors to understand the quality and 
effectiveness of career readiness data. The survey was conducted in partnership with the Data 
Quality Campaign; the Workforce Data Quality Campaign, a project of the National Skills Coalition; 
Education Strategy Group; and the Council of Chief State School Officers. It was generously funded 
by JPMorgan Chase & Co. through the New Skills for Youth initiative. A total of 51 State Directors 
responded to the survey, representing 48 states, two territories and the District of Columbia. 


=. 


The survey illuminated shortcomings across state CTE data systems and lends new urgency to 
states’ efforts to improve data quality and use. The survey found that: 


e States are hesitant to use their data for high-stakes decisionmaking. The most common use 
of career readiness data across a variety of measures is to inform technical assistance to local 
programs and institutions or inform program improvement efforts. Using data to provide target- 
ed support to local institutions is certainly a best practice. However, the survey shows that states 
are opting for a “carrot” rather than a “stick” approach when it comes to program improvement. 
While 71 percent of State Directors report using two or more measures of career readiness to 
inform technical assistance and program improvement at the secondary level, only 43 percent 
report using at least two measures to transform or phase out CTE programs. At the postsecond- 
ary level, 57 percent of states report using two or more measures of career readiness to inform 
technical assistance and program improvement, whereas only 35 percent report the same for 
transforming or phasing out programs. This finding indicates that states either are not maximiz- 
ing their data to hold local institutions accountable for learner success or do not trust the quality 
of their data enough to make high-stakes decisions. 
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e CTE data systems are not aligned across 
the secondary, postsecondary and work- 
force sectors. CTE sits at the intersection of 
secondary education, postsecondary edu- 
cation and the workforce — yet each sector 
often operates in its own silo. All too often, 
states use different data systems, measures 
and collection cycles for these sectors, even 
when they serve the same population of 
learners. Alignment is strongest at the sec- 


ondary level, but even in that sector only 49 


percent of State Directors report that their 


CTE data systems are mostly or fully aligned 
with secondary data systems. Twenty-eight 


percent of State Directors report that their 


CTE data systems are mostly or fully aligned 
with postsecondary data systems, and only 18 
percent report the same for workforce data. 


As State Directors are increasingly called on 
to align and embed CTE within cross-sector 
career pathways, there will be an urgent need 
to improve and align data systems to better 
monitor and evaluate learner outcomes. 
According to State Directors, the barriers to 
improving data systems are more a function 
of technology than leadership and political 
will. Eighty-eight percent of State Directors 
report that separate data systems or inconsis- 
tent definitions are a major barrier to align- 
ment. Establishing cross-sector data sharing 
partnerships requires time, money and com- 


mitment across state agencies, but it can pay 
off immensely by improving the reliability and 


validity of data, reducing the reporting burden 


on local institutions, and providing better 
access to information across sectors. 
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e Many states rely on self-reported 
measures of career readiness and do not 
often use rigorous validation processes. 
This report examines four primary measures 
of career readiness: completion of a work- 
based learning experience; attainment of a 
recognized postsecondary credential, in- 
cluding industry-recognized credentials and 
postsecondary degrees; completion of dual 
or concurrent enrollment; and successful 
transition to further education, employment 
or the military. A handful of states have adopt- The survey lends 


ed rigorous processes to collect and validate new urgency to 


these measures. For example, 12 percent of states’ efforts to 


states collect credential attainment data for improve data 


secondary students directly from credential quality and use. 
providers, and 10 percent of states do so for 
postsecondary students. But many continue 
to rely on outdated or unreliable methods. At 
the secondary level, 61 percent of states use 
student surveys, which have notoriously low 
response rates and are difficult to validate, 

to identify whether learners secure post- 
program employment. Thirty-three 

percent use the same method at the 
postsecondary level. 


That said, several states have successfully 
restructured and improved their data sys- 
tems, putting in the time and effort to secure 
meaningful partnerships, collect and validate 
high-quality data, align data systems across 
sectors, and leverage career readiness data to 
transform career pathways and improve out- 
comes for young learners. Their examples are 
highlighted throughout this report to provide a 
roadmap for other states. 


Executive Summary 


At this point in time, State Directors do not have enough confidence in their data to use them in 
making decisions about data quality and other initiatives. But states have cause for optimism. With 


the reauthorization of the Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) 
— retitled the Strengthening Career and Technical Education for the 21st Century Act (Perkins V) 

— in 2018, states are charged to redesign and restructure their strategic visions and plans for CTE, 
many for the first time in more than a decade. This opportunity gives states the chance to realign 


and improve their CTE data systems in a truly impactful way. Perkins V includes new requirements 
for accountability and reporting, data collection and data-driven decisionmaking, and states 


should seize the opportunity to examine and improve the quality and effectiveness of their data. 
This work includes: 


e Realigning state data systems around a unifying state vision for career readiness; 
e Auditing data collection processes and identifying opportunities to improve data quality; 


e Committing to transparency and using CTE data to ensure equity across all learner populations; 
and 


e Taking bold steps to connect data with program approval and funding decisions. 


This report unpacks in more detail key trends from the State of CTE survey and charts a path for 


states to improve their CTE data systems. 
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Career Technical Education (CTE) prepares each 
and every learner to have the confidence and 
skills to enter the world of work and ensures 
that employers have a pipeline of talent to close 
critical skill gaps. To meet these ends, all stake- 
holders — particularly those in the secondary, 
postsecondary and workforce sectors — must 
work together to put learner success first. 


Critical to achieving this vision is a strong, 
inter-connected data system that can fol- 
low learners as they move from secondary 
to postsecondary education and into the 


workforce. Data systems are powerful tools 
that can improve the quality of and access to 
meaningful career readiness opportunities. 
State CTE Directors can use data to identi- 

fy and close equity gaps in CTE programs, 
continuously improve CTE programs and 
programs of study, evaluate learner outcomes 
and program impact, and hold institutions 
accountable for learner success. 


How effective are state CTE data systems at 
meeting these objectives? To address this 
question, Advance CTE — in partnership with 

the Data Quality Campaign; the Workforce Data 
Quality Campaign, a project of the National 


Oo 


Skills Coalition; Education Strategy Group; and 
the Council of Chief State School Officers and 
with generous funding from JPMorgan Chase & 
Co. — conducted a national survey in fall 2018. 
A total of 51 State Directors responded to the 


survey, representing 48 states, two territories 


and the District of Columbia.t This report draws 
on information from that survey to examine the 
quality of state CTE data systems and unpack 
common challenges and promising solutions 
from across the country. 
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Overall, 86 percent of State Directors say that 
improving and enhancing their state CTE data 
systems is a top priority. Yet, the data systems 
they currently use are insufficient to meet 21st 
century educational needs. Only 45 percent of 
State CTE Directors report that their CTE data 
systems provide the information they need to 
assist in making decisions about CTE program 
quality and initiatives at both the secondary 
and postsecondary levels. 


Why are State Directors unable to access the 
data they need? For one, state CTE data systems 
are not sufficiently aligned with data systems in 
other sectors, making following learners as they 
progress from high school to college and into 
the workforce challenging. Addressing this issue 


iS an urgent priority. As state and federal policy- 
makers recognize the value of postsecondary 
education and the demand for a highly skilled 
workforce, they are pushing secondary, post- 
secondary and workforce leaders to work more 
collaboratively to achieve common goals. As of 
2018, 40 states had adopted measures of career 
readiness in their state or federal accountability 
systems for high school students.* And nearly 
every state has set goals around postsecondary 
credential attainment, leading statewide strate- 
gies to scale career pathways that culminate in a 
credential of value.* These strategies cut across 
sectors and require partnership and data sharing 
to effectively measure success. 


Another barrier is the quality of CTE and 
career readiness data. States often rely on 
self-reported information to identify learners 
who earn industry-recognized credentials, 
complete a work-based learning experience, 
earn postsecondary credit in high school, or 
go on to further education or employment. 


Overall, 86 percent 
of State CTE Directors 
say that improving 
and enhancing their 
state CTE data systems 
is a top priority. 


Introduction 


FIGURE 1: THE STATE OF CTE DATA SYSTEMS 


The current CTE data system in my state provides the information needed to 
assist in making decisions about postsecondary program quality and CTE initiatives 


14% 37% 22% 16% 12% 


The current CTE data system in my state provides the information needed to 
assist in making decisions about secondary program quality and CTE initiatives 


16% 


Improving and enhancing the CTE data system is a top priority for my state 


59% 27% 8% 4%2% 


HN STRONGLY AGREE [AGREE [J NEITHERAGREENOR DISAGREE [DISAGREE [STRONGLY DISAGREE 


Without proper validation protocols, stakeholders cannot make fully informed decisions about 
policy, program improvement and funding. 


This report comes at an opportune time. With the reauthorization of the Carl D. Perkins Career 
and Technical Education Act in July 2018, states will need to make adjustments to their CTE data 
systems to comply with new federal requirements. Specifically, Perkins V: 


e Includes a new definition for CTE concentrators at the secondary and postsecondary levels that 


will require most states to adjust their methodology for counting these individuals; 
e Requires states to disaggregate CTE performance and participation by new special populations; 


e Shifts performance accountability metrics and directs states to define their own measures of 
secondary CTE program quality; 


e Requires local recipients to use data to inform their comprehensive local needs assessment and 
their local application; and 


e Has a renewed focus on equity in both state and local planning.* 


While Perkins IV successfully pushed states to collect and report performance data for CTE students, 
often for the first time, Perkins V aims for a renewed focused on data-driven decisionmaking and 
expanding opportunity and access. At the same time, CTE is being challenged to meet the needs of 
a rapidly growing economy. States have an opportunity under Perkins V to enhance the quality of 
their data systems and leverage CTE data to improve program quality and secure equitable out- 
comes for all learners. As states prepare new visions and plans for the future of career preparation 

— looking beyond CTE to K-12 education, workforce development and more — they will need to 
enhance their data systems accordingly. But they must seize the opportunity and make the most of 
Perkins V implementation before it is too late. 
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How Are States Using Their 
Career Readiness Data? 


Common Measures of Career Readiness 


Learners can demonstrate career readiness in 
many ways, from assessments of learning to 
documented work experience and industry 
mentorship. States define and measure career 
readiness through different approaches, some 
outlined specifically in federal statute and some 
by states themselves. Table 1 demonstrates 
different requirements for measuring success 
outcomes across three inter-connected federal 


programs: Perkins IV and V, the Every Student 
Succeeds Act (ESSA), and the Workforce Inno- 
vation and Opportunity Act (WIOA). While each 


TABLE 1: POPULATION AND ACCOUNTABILITY REQUIREMENTS 
FOR FEDERAL EDUCATION AND WORKFORCE DEVELOPMENT PROGRAMS 


of these laws, and the state agencies responsi- 
ble for delivering their respective programs, 
operate in disparate sectors, they often serve 
the same population of students and have 
similar measures of success. For example, 
most states have chosen to measure career 
readiness in their ESSA accountability systems, 
signaling an expanded focus on career read- 
iness beyond CTE. In these states, common 
measures can provide an opportunity to align 
goals and coordinate service delivery across 
these programs. 


Secondary and postsecondary CTE 
participants and concentrators 


PERKINS IV 


¢ Secondary academic attainment 
¢ Technical skill attainment (secondary and postsecondary) 
¢ Secondary school completion 
« Secondary graduation rate 
¢ Student placement (secondary and postsecondary) 
¢ Non-traditional participation and completion 

(secondary and postsecondary) 
¢ Postsecondary credential, certificate or diploma attainment 
e Postsecondary student retention or transfer 


Same as Perkins IV 


PERKINS V 


« Secondary graduation rate 
« Secondary academic proficiency 

« Postsecondary placement (secondary) 

« Non-traditional enrollment (secondary and postsecondary) 
e Secondary program quality 

« Postsecondary retention and placement 

« Postsecondary credential, certificate or diploma attainment 


ESSA 


All public elementary and secondary students 


« Academic achievement 

e Another academic indicator 

* Graduation rate (high school only) 

English language proficiency 

A state-determined indicator of school quality or student success 


WIOA program participants such as 
out-of-school youths ages 16-24, in-school 
youths ages 14-21, adults with limited 
English proficiency, dislocated workers, 
individuals with disabilities and job seekers 


WIOA 


edian earnings 

Credential attainment 

easurable skill gain 

« Effectiveness in serving employers 
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Employment or education (second quarter and fourth quarter after exit) 


How Are States Using Their Career Readiness Data? 


This report examines four primary measures of career readiness, which are commonly used in 
federal and state reporting and are often discussed as signals of career readiness at the secondary 
and postsecondary levels. These four measures include: 


rm 
[7] e Completion of a work-based learning experience; 


AS e Attainment of a recognized postsecondary credential, including industry-recognized 
credentials and postsecondary degrees; 


= e Completion of dual or concurrent enrollment; and 
C~ e Successful transition to further education, employment or the military. 


According to State Directors, nearly every state can collect learner-level information on these 
measures at either the secondary or postsecondary level. However, these measures are stronger 


at the secondary than at the postsecondary level, and, as the rest of this report will examine, these 
measures are not always valid and high quality. 


FIGURE 2: PERCENTAGE OF STATES THAT CAN COLLECT LEARNER-LEVEL 


DATA ON A VARIETY OF CAREER READINESS MEASURES 


Industry-Recognized Credential Attainment (Secondary) 96% 


Further Education or Training (Secondary) 94% 


Dual/Concurrent Enrollment 92% 
Post-Program Employment (Secondary) Te) 
Secondary Work-Based Learning 88% 
Post-Program Employment (Postsecondary) 84% 
Further Education or Training (Postsecondary) 82% 
Industry-Recognized Credential Attainment (Postsecondary) VEY 


Transfers between 2-Yr CTE Programs and 4-Yr Programs 47% 


0% 25% 50% 75% 100% 


Hl SECONDARY MEASURES [J POSTSECONDARY MEASURES 
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States are correct to measure and incentivize Disaggregating Career Readiness Data 


these activities, and a growing body of research ; 
7 To use data effectively, states must be able to 
has demonstrated positive economic outcomes . : 
disaggregate measures of career readiness by 
for learners who attain these milestones in their ; 
; as. Career Cluster®, program and sub-population 
youth. The Brookings Institution finds a cor- ; ; 
to better identify patterns in enrollment and 
relation between work-based learning and job 
a a performance. Most states are well prepared to 
quality later in life.° Numerous studies find that ; ; see 
a examine their data with precision, in part 
learners who earn college credit in high school ; 
eas ene au : because of federal reporting requirements. 
are more likely to enroll and persist in postsec- ; 
- 7 P Perkins IV directed states to disaggregate CTE 
ondary education.’ ® But counting learners who oe 
performance indicators by Career Cluster; 
complete these milestones is only the first step. - 
Stee sould id oe 4 gender; race/ethnicity; and special popula- 
ates should consider a variety of career readi- ane is Soi penne 
/ z tions, which include individuals with disabilities, 
ness experiences throughout a learner's career : . eaten 
Re F nceeen ae . economically disadvantaged individuals and 
athway and ensure that data are hi uality, ; ; 
P : eee other populations. Perkins V further aligns these 


expectations with ESSA and WIOA; adopts addi- 
tional sub-populations for reporting; and directs 


aligned across sectors, and able to be disaggre- 
gated by different learner populations so states 
can use them effectively to improve program . 
states, to the extent practicable, to disaggregate 
quality and ensure equitable learner success. bg: 
performance indicators by CTE program of 


study in addition to Career Cluster. 


TABLE 2: STATES THAT CAN DISAGGREGATE CAREER READINESS MEASURES 


BY CAREER 
CLUSTER 
BY CTE PROGRAM 
OR PROGRAM 
OF STUDY 
BY RACE/ 
ETHNICITY 
BY GENDER 
1) MND) AVAL BOY as) 
WITH DISABILITIES 
BY ECONOMICALLY 
DISADVANTAGED 
INDIVIDUALS 


ui 
x 


Secondary Work-Based Learning 


Industry-Recognized Credential Attainment 
(Secondary) 


Industry-Recognized Credential Attainment 
(Postsecondary) 


Dual/Concurrent Enrollment 


Transfers Between 2-Yr CTE Programs and 
4-Yr Programs 


Post-Program Employment (Secondary) 


Post-Program Employment (Postsecondary) 


Further Education or Training (Secondary) 


Further Education or Training (Postsecondary) 63% 61% 


MM 75%-100% OF STATES [ 50%-74% OF STATES 25%-49% OF STATES [ O%-24% OF STATES 
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How Are States Using Their Career Readiness Data? 


Using data to create 
change means 
embedding data in 
every step of the 
decisionmaking 
process, from adopting 
state policy to improving 
program quality. 
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As Table 2 shows, most states can disaggregate their data by Career Cluster, CTE program of study 
and additional sub-populations for a variety of career readiness measures. The only exceptions are 

for measures at the postsecondary level. Less than half of states can disaggregate successful trans- 
fers between postsecondary two-year CTE programs and four-year programs, and only 43 percent 
of states report that they can disaggregate the CTE program or program of study related to creden- 
tials attained at the postsecondary level. 


The measures states are best prepared to disaggregate are secondary attainment of industry- 
recognized credentials, dual or concurrent enrollment, post-program employment for postsecond- 


ary learners, and post-program enrollment in further education or training for secondary learners. 
Many states have chosen to count industry-recognized credential attainment and dual or concur- 
rent enrollment in their state and federal high school accountability systems, which may be why 
such large percentages of states can disaggregate these measures. The fact that so many states 
can disaggregate post-program outcomes may be a result of reporting requirements under Perkins 
lV. While states can report these measures as a "“Meta-indicator” — meaning they can include in the 
numerator any learner who goes on to further education or training, employment or military service 
— they are required to disaggregate this meta-indicator by sub-population. 


That so many states can disaggregate measures of career readiness is promising. However, this 
S 


ituation makes focusing on improving the quality and validity of the data all the more important 


for states. Even if states can break down their data on post-program employment, this measure Is 
Vj 


rtually meaningless if it is derived from self-reported surveys with a low response rate. And states 


still have work to do to ensure that they can disaggregate data for work-based learning, industry- 


recognized credential attainment, and other measures of career readiness by all program areas 


and sub-populations. 
Using Career Readiness Data to Effect Change 


Data are effective only if they are used strategically to create change. Using data to create change 
means embedding data in every step of the decisionmaking process, from adopting state policy to 


improving program quality. If data are high quality, valid and aligned longitudinally, they can provide 
color and nuance to a state’s CTE programs. State leaders should make sure to wield career readi- 
ness data like a scalpel rather than a hammer, delivering necessary supports with surgical precision 
to ensure that each learner can access and succeed in high-quality career pathways. 


This report examines how states use the four measures of career readiness identified in the sur- 
vey: completion of a work-based learning experience; attainment of a recognized postsecondary 
credential, including industry-recognized credentials and postsecondary degrees; completion of 
dual or concurrent enrollment; and successful transition to further education, employment or the 
military. Figure 3 depicts the proportion of states that are using at least two of these four measures 
of career readiness to influence decisionmaking at the secondary or postsecondary level. See 
Appendix A for a further breakdown of how states use each of the four measures. 
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FIGURE 3: COMMON USES OF CAREER READINESS DATA (SECONDARY AND POSTSECONDARY) 


Percentage of states using at least two measures of career readiness to ... 


Influence state or 
' federal funding 
: decisions 


Influence decisions to 
transform or phase out 
CTE programs 


35% 37% 


0% 100% 0% 100% 


LEAST COMMON USE 


Informing Technical Assistance 


and Program Improvement Efforts 


One of the primary levers state leaders can 

use to influence program quality and equity 

iS technical assistance and supports to local 
leaders. While many decisions about the design 
and delivery of CTE are made at the regional or 
institutional level, state leaders can help improve 
program quality and equity by sharing promising 
practices from across the state, providing profes- 
sional development for educators and adminis- 
trators, and directing resources toward program 
improvement efforts. Using data to inform and 
prioritize these supports can ensure that local 
investments have the maximum impact. 


According to State Directors, this use of career 
readiness data is the most common. As Figure 3 
shows, 71 percent of states use at least two out 
of the four career readiness measures included 
in the survey to inform technical assistance and 
program improvement efforts at the secondary 
level. Fifty-seven percent of states report the 
same at the postsecondary level. 
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Ml SECONDARY 


Include in publicly 
reported data 

or in the state 
accountability 
system 


Inform state policy 
and planning 


0% 100% 0% 


100% 


Ml POSTSECONDARY 


One approach is Idaho's CTE Program Quality 
Initiative, which provides program incentive and 
technical assistance funding to reward outstand- 
ing program performance as well as help local 
program leaders improve performance ona 
variety of state-identified performance mea- 
sures. For the technical assistance funding, 


the Division of Career & Technical Education 
examines local program data annually to iden- 
tify school districts that are below target and 
then reaches out to these programs to offer 
resources and technical assistance. In 2018, 


daho received 94 applications and distributed 
44 awards to support program improvement 


efforts. The Legislature has provided some 
investment to support the initiative the past two 
years, and the Appropriations Committee voted 
for additional funds during the 2019 legislative 
session, providing sustainable incentive fund- 
ing in the future to encourage all programs to 
achieve excellence. 


Inform technical 
assistance to 
institutions and 
programs or inform 
local program 
improvement efforts 


57% 


0% 100% 


MOST COMMON USE 


Idaho CTE Program 
Quality Initiative 
Performance Measures 


e Technical skill 
assessment success 


¢ Workplace readiness 
assessment success 


e Active involvement 
in career technical 
student organizations 
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Informing State Policy and Planning 


State leaders can also use their data to inform state policy and planning. Good policy is evidence 
based, drawing on data to scale effective strategies, address barriers and improve ineffective pro- 
grams. States should make examining career readiness data a common practice and an expectation 
for all major decisions. This use of career readiness data is the second most common across the 
states, according to the survey. Sixty-seven percent of states report using at least two out of the four 
career readiness measures examined in the survey to inform state policy and planning at the sec- 
ondary level, and 53 percent report using at least two out of four at the postsecondary level. 


Arkansas, for example, partnered with the Thomas B. Fordham Institute and the University 


of Connecticut in 2016 to examine post-program outcomes for CTE students.? The Arkansas 


Research Center, operating under a U.S. Department of Education grant, created a statewide 


longitudinal data system that provided learner-level data for the analysis. The report found that 

post-program outcomes — including high school graduation, enrollment in a college or univer- 
sity, employment and earnings — were notably higher for learners who concentrated in a CTE 
program of study than for those who did not. State leaders in Arkansas have used this report to 


demonstrate to state policymakers the benefit of completing a sequence of high-quality CT 
courses within a program of study. 


Similarly, Kentucky uses information about dual credit earners to inform new investments and 
policy changes. In 2016, Gov. Matt Bevin launched the Dual Credit Scholarship Program to provide 


funding for high school seniors to take up to two college courses.’° Shortly thereafter, the Legis- 


lature expanded the initiative to include high school juniors. State leaders in Kentucky discovered 
that learners who complete dual credit are 30 percent more likely to persist in their second year of 
college, making a clear case for continued investment. Yet as dual credit grows in popularity, 
Kentucky has discovered that more and more students are participating in “random acts of dual 
credit” that are disconnected from their programs of study. As a result, the commonwealth is now 
working to set up guardrails to ensure that learners’ course-taking patterns are more closely aligned 
to their programs of study. 


Including Career Readiness Measures in Publicly Reported Data or Accountability Systems 


States also use career readiness data for public reporting or accountability purposes. Sixty-one 
percent of states report using at least two measures of career readiness for public reporting or 
accountability at the secondary level. However, this percentage is much lower at the postsecondary 
level, with only 37 percent of states reporting the same. Making career readiness data available to 
the public empowers stakeholders to identify priorities in their communities and become stronger 


advocates. If an institution is underperforming or failing to serve all learners equitably, then learners, 
parents and community members deserve to know so they can take action. Building such measures 


into state accountability systems for secondary and postsecondary education puts weight behind 
the data and compels institutions to focus their capacity on improving program quality and equity. 
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While many states have committed to start 
reporting their CTE and career readiness data, 
this strategy is still relatively nascent. Several 
states are reporting measures for the first time 
through their ESSA report cards, and others are 
still working through and refining their business 
rules. Ohio, however, first started publishing an 
annual CTE report card for schools and districts 
in 2013. The report card included a variety of 
indicators of career readiness and learner 
performance such as technical skill attainment, 
graduation rate, attainment of an industry- 
recognized credential, earning remediation-free 
scores on the ACT or SAT and post-program 
outcomes." These report cards not only provid- 
ed school and district leaders the information 
they needed to improve their programs but also 
empowered stakeholders in the community to 
be better advocates, hold institutions in their 
community accountable and celebrate success- 
es. The CTE report card data have now been 
integrated into the statewide report cards. 


One priority that state policymakers in Georgia 


have elevated is participation in work-based 
earning and youth apprenticeships. The state 
developed a comprehensive work-based 
earning policy manual, which outlines the 
core requirements and structure for different 
types of experiences, and designed a virtual 


information hub to share critical resources with 


earners and employers. The website tracks 
participation in work-based learning and allows 
viewers to examine data by type, credits earned 
and Career Cluster of focus.” 
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Using Data to Transform Career Pathways 


These three approaches — informing technical 
assistance or program improvement, informing 
state policy and planning, and reporting career 
readiness data to the public — are effective 
strategies but might not in themselves be trans- 
formative. To achieve quality and equity state- 
wide, states must be able to rely on their data 


to make high-stakes decisions about funding or 
phasing out underperforming programs. 


Only 47 percent of states report using at least 


two measures of career readiness to make 
funding decisions at the secondary level and 
37 percent at the postsecondary level. These 


states should be recognized for their efforts 
to align resources and funding with program 


improvement, a critical strategy for transforming 
career pathways quality. Yet with less than half 
of states using data in this way, states appear to 


Program Enrollment 


Internships 


Employability Skill Students 
Cooperative Education 


Great Promise 


Coo es 


2000 4000 6000 


Total Credits Earned 


[COZ] Credits Earned Year-to-Year 


34000 
32000 
30000 
28000 
26000 
24000 
22000 
20000 
18000 


8000 


2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 


Georgia’s work-based learning dashboard. 
Retrieved from https://gawbl.org/by-the-numbers 
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be reluctant to rely on their data for high-stakes decisions such as connecting funding with program 
quality. Often, state CTE offices are organized in such a way that decisions about program quality 
and funding are disconnected. These decisions should be better coordinated to ensure that outdat- 


ed and low-quality programs are not sustained. 


Only 43 percent of states use two or more career readiness measures to transform or phase out 
programs at the secondary level and 35 percent at the postsecondary level. Again, states may 

be hesitant to make high-stakes decisions based on data they do not fully trust. Having reliable, 
high-quality data that all stakeholders can understand and rely on allows for difficult conversations 
and creates the opportunity for meaningful, transformative work. Such decisions are not easy to 
make, but they must be informed by data to ensure that all programs are high quality and that each 
learner is able to access and succeed in his or her career pathway of choice. 


One state that has taken a bold but transparent approach to program phase-out is Wisconsin. 

The Wisconsin Technical College System's (WTCS) program suspension policy states that associate 
degree and other sub-baccalaureate programs may be subject to program phase-out if they fail to 
meet goals for enrollment, retention, placement and other outcomes. This process is meant to 
be collaborative between the institution and WTCS. Before a program is placed on suspension, the 
education directors and assistant vice president of instruction at WTCS reach out to the program 
dean to identify opportunities for improvement. Often, faculty and staff at the institution are willing 
to place a program on suspension to free up resources and capacity for newer, more relevant offer- 
ings. Once a program has been placed on suspension, a phase-out period accommodates learners 
who are already enrolled, but no new students are permitted to enroll. After three years of suspen- 
sion, the discontinuation process Is triggered, and the program is terminated. 
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Career Readiness Measures 


One of the likely reasons State Directors are personalized learning plan that documents 


not fully leveraging their career readiness data goals; skill development; and details about the 
to transform career pathways is because they placement such as the name of the employer, 


do not trust the quality and validity of their data. — the duration of the experience, and whether the 


mproving the quality of career readiness mea- 
sures is an urgent imperative for states, and 
there is a lot of opportunity for improvement. 
While many states have developed processes 
to collect and disaggregate data for state and 


£ 


ederal reporting, these methods are often 
archaic, inefficient and unreliable. 


How States Collect Data on 
Career Readiness 


Overall, states do not appear to have a consis- 
tent approach to measuring career readiness 
data, as Table 3 shows. The most common 
approach is to report data through a state-level 
student information system or through a state- 
wide longitudinal data system (SLDS). Ninety 
percent of states measure dual or concurrent 
enrollment, 67 percent collect secondary 
industry-recognized credential attainment, 

and 61 percent collect secondary work-based 
learning data through a statewide data system. 
Yet fewer states use this method to collect other 
measures, such as post-program employment 
or pursuit of further education or training, and 
data collection overall does not appear to be as 
strong at the postsecondary level. 


One example of a state that collects learner- 
level career readiness data at the state level is 
Tennessee, which uses a separate statewide 
portal for educators to submit information about 
learners who are placed in work-based learning. 
Each learner is required to have a complete 
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experience is paid or unpaid. While the work- 


based learning portal is 
not directly connected 
to Tennessee's student 
record database, state 


leaders in the Depart- 
ment of Education can 
do a backend match to 
link the data with the 
learner's profile. 


Some states, but not 

as many, use a locally 
sourced student in- 
formation system or 

a separate statewide 
Perkins data collection to 
gather learner-level data. 
Far fewer partner with 
other agencies or orga- 
nizations, such as the 
National Student Clear- 
inghouse, to measure 
career readiness. 


It is worth noting that 
States rely heavily on 
self-reported informa- 
tion for post-program 
outcomes measures. 
Sixty-one percent of 


states report that they 
use surveys to learn 


CTE and SLDSs: 
A Natural Fit 


More and more, state leaders are 
working to connect learner-level 
data at the state level through 
SLDSs. While SLDSs historically have 
included K-12 data, many have been 
expanded to include learner-level 
data from preschool through college 
and into the workforce (commonly 
known as P20W systems). 


Setting up a cross-agency data sys- 
tem requires extensive partnership, 
trust and commitment of resources 
across sectors, but the impact is 
often worth the effort. States with 
strong SLDSs can use their data 

to answer critical questions about 
which courses and experiences are 
most strongly correlated with 
graduate success. 


States should make sure their SLDSs 
include CTE data and other learner- 
level information related to career 
readiness. Often, CTE data are mea- 
sured separately from data within the 
same sector. Ensuring these data are 
included in the SLDS facilitates better 
alignment within the sector, ensures 
that other experiences that are not 
captured through traditional CTE data 
collections are measured, and allows 
state policymakers to evaluate out- 
comes for learners who complete CTE 
or other career-focused experiences. 
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TABLE 3: HOW STATES COLLECT LEARNER-LEVEL CAREER READINESS DATA 


COLLECTED THROUGH A STATEWIDE 
STUDENT INFORMATION SYSTEM OR SLDS* 
COLLECTED BY INDIVIDUALS 
OR INSTITUTIONS THROUGH SURVEYS 
COLLECTED THROUGH A LOCALLY SOURCED 
STUDENT INFORMATION SYSTEM 
COLLECTED THROUGH A SEPARATE 
PERKINS DATA COLLECTION 


co) 
ney 
3s 


Secondary Work-Based Learning 8% 10% 


Industry-Recognized Credential 


t:) 
Attainment (Secondary) i 


24% 


Industry-Recognized Credential 


Attainment (Postsecondary) 10% 24% 24% 


Dual/Concurrent Enrollment 2% 6% 


Transfers Between 2-Yr CTE 


9, 9 [-) 
Programs and 4-Yr Programs 2% 14% 2% 


Post-Program Employment 


(Secondary) 61% 


COLLECTED THROUGH AN AGREEMENT 
WITH THE STATE DEPARTMENT OF 
LABOR OR OTHER AGENCY 
THE NATIONAL STUDENT CLEARINGHOUSE 
COLLECTED THROUGH AN AGREEMENT 
WITH CREDENTIALING BODIES 
PERCENTAGES OF STATES 
THAT ARE ABLE TO COLLECT 


COLLECTED THROUGH AN AGREEMENT WITH 


2 
SS 


Post-Program Employment 
(Postsecondary) 


Further Education or Training 
(Secondary) 


Further Education or Training 
(Postsecondary) 


*State Longitudinal Data System MM 75%-100% OF STATES [EN 50%-74% OF STATES 
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about post-program employment for secondary learners, and 33 

t to | t {- l tf 
percent use surveys to learn about post-program employment for Take thenext step) DOC has develoced 
roadmaps to help state leaders link K-12 and 
student surveys administered by the school, school district or insti- workforce data as well as secondary and 
tution, though some institutions rely on methods such as tracking postsecondary data. These instructive tools 
provide recommendations and promising 
examples to help states design and build 


postsecondary learners. This information is often collected through 


students down on LinkedIn or Facebook to find out where they end 


up. This approach not only leaves significant room for response connected data systems.” 

error and misreporting but also leaves institutions susceptible to low 

response rates. When institutions can track down only a fraction of CE io: CS 
their program graduates, state leaders are not able to look at a repre- Workforce Bats Linkages Laage eC eeMrececoey 


Key Focus Areas to Ensure Quality Implementation 


sentative sample of program participants. 


At the postsecondary level, a higher percentage of states 


(45 percent) have successfully partnered with other state 
agencies to access employment data, but this proportion still rep- 
resents less than half of survey respondents. The percentages are 
slightly better, but not much, for measuring post-program 


enrollment in further education or training: 55 percent of states 
use Surveys at the secondary level, and 31 percent use them at 
the postsecondary level. 


mproving the collection of outcome measures is an area of urgent need for State Directors. 
Decisionmakers must be able to see if learners are gaining meaningful employment or continuing 


their education once they exit their program, yet all too many states rely on sub-standard methods 
for collecting this information. When it comes to measuring post-program outcomes, states can 


improve the quality and efficiency of their data systems by thinking smarter, not harder. Rather than 
relying on self-reported data or student surveys — which are burdensome to administer and provide 
an inaccurate portrait of learner outcomes — states should invest in data systems based on strong 


partnerships and cross-agency data governance that allow them to leverage administrative records 
from other sectors to evaluate learner success. While decisions about cross-sector data sharing and 
governance are not in the purview of most State Directors, they can help start the conversation by 
critically examining the quality of their data, calling for access to better information, and demonstrat- 
ing the impact that improving data quality can have on CTE program quality across the state. 


State Directors should also consider whether other partners in their state, or even in other states, 
can supplement their learner data and validate the information they have on record. For example, 
states can partner with credential providers to access administrative records for learners who sit for 


and earn a credential of value. Only 12 percent of states have established such data sharing part- 


nerships at the secondary level and only 10 percent at the postsecondary level, but agreements in 
these states cover only a fraction of the industry-recognized credentials available to and attained by 
learners. For the vast majority of states, information about industry-recognized credential attainment 


is reported by local schools and institutions without verifying that the learner received the credential 
from the provider. 
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Take the next step: In 2017, the Certification 
Data Exchange Project — a partnership of 
state leaders, credential providers and 
researchers — developed a five-step process 
for obtaining credential attainment data: 


e Establish the data sharing agreement; 

e Estimate bandwidth and allocate resources; 
e Facilitate the transfer of data; 

e Perform the analysis; and 


e Present the results using common 
templates. 


The full report describes lessons learned 
from the seven participating states, which 
were able to access nearly 100,000 individual 
credential attainment records collectively. 
The report also includes sample data sharing 
agreements and reporting templates.*© 


Mes 


Connecting Industry-recognized 
Certification Data to Educati 
and Workforce Outcome: 
Mi 


DW 


heir state data systems alone. 


North Carolina is one state that has begun collecting 
industry-recognized credential attainment data directly from 
credentialing bodies. The state recently launched an incentive 
program to reward teachers for each student who successfully 
earns a state-recognized credential’ To accurately identify 
these individuals, the Department of Public Instruction set up 
data sharing agreements to gather data directly from creden- 
tial providers and has managed to successfully obtain data for 
more than half of credentials earned by high school students 
in the state. The remaining credentials are reported by teach- 
ers and certified by instructional leads. To match learner data 
between the secondary data system and the credential provid- 
ers database, North Carolina uses the learner's name, location, 
date of birth and other unique information. 


Likewise, the Minnesota State Colleges and Universities system is 


collaborating with industry-recognized credential providers across 
a variety of program areas to obtain credential attainment data. 
State leaders have gained access to certification data from cre- 
dential providers for emergency medical technician, automotive 
service, occupational therapy and other programs. However, the 
quality of the data varies from partner to partner. Some provide 
aggregated totals while others provide learner-level data. 


Another approach to collecting data on career readiness is partnering with the National Student 
Clearinghouse, a non-profit organization supported by partnerships with more than 3,000 post- 
secondary institutions, to gather information on enrollment in further education, transfer between 
postsecondary programs, completion of dual or concurrent enrollment and credential attainment. 
ut less than half of states report using the National Student Clearinghouse to collect data for 
hese measures. One advantage to using the Clearinghouse for this information is that states can 
rack students who attend a college or university in another state, something they cannot do with 


Clearly, many states have an opportunity to improve methods of data collection, moving away 
from self-reported information and toward more reliable sources of information. 


How States Validate Career Readiness Measures 


In addition to improving methods of collection, states should adopt processes and protocols to 


review, validate and certify career readiness data. These processes and protocols can improve 


data quality by reducing errors and ensure that state policymakers can trust the accuracy of the 


data states report. States can take a number of different approaches to validating their career read- 


iness data — such as examining data for anomalies or verifying information with other datasets — 


but at a minimum states should have some validation protocol in place. 


18 


THE STATE OF CAREER TECHNICAL EDUCATION: 


Overall, states do not appear to be using robust 
processes and protocols for validating career 
readiness data. The most common approach 

is to conduct a state audit or review, though 
how involved these reviews are is unclear. 
Nearly as many states indicate they have no 
state process at all to review or validate career 
readiness measures, particularly for measuring 
work-based learning and industry-recognized 
credential attainment. 


To the extent practicable, states should verify 
career readiness data with other agencies, 
organizations and partners to confirm that the 


reported information is accurate. This process 


means having employers validate that a learn- 
er has completed a work-based learning expe- 
rience, verifying with postsecondary records 
that a learner has earned transcripted credit as 
a result of dual or concurrent enrollment, or 
partnering with credentialing bodies to con- 


firm that learners have sat for and earned an 


ndustry-recognized credential of value. 


In Arkansas, schools are required to get em- 
ployer validation when participants complete a 
work-based learning assignment. Before their 


placement, learners identify specific skills they 
want to develop through their experience and 


how they want to be assessed by their worksite 
mentor. Employers are required to submit docu- 
mentation to validate that the work-based learn- 
ing experience was completed as described. 


In South Dakota, the primary way high school 
students earn dual credit is by attending classes 
at Board of Regents schools and public tech- 
nical institutes at a reduced tuition rate. Infor- 
mation on postsecondary credit attainment 

for these students is received directly from the 


institution awarding the credit. Massachusetts 
validates data among the school district's 
records, the National Student Clearinghouse 
and the higher education sector in the state to 
confirm that learners who are reported as earn- 
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ing postsecondary credit are also recognized as 
earning credit within the postsecondary sector. 


Another approach states can take to validate 
career readiness data is building error rules or 
establishing thresholds for identifying anomalies 


in their data. In Wisconsin, validations are built 
into the secondary and postsecondary data 


collection system for dual or concurrent en- 
rollment to trigger any incomplete information. 
Tt 

at the input stage. In Virginia, the Department of 


23 


is Validation ensures that errors are identified 


Education uses reports from credential providers 


to verify and audit credential attainment data. 
The department conducts an electronic and 
manual analysis to identify discrepancies, such 
as irregular variances in the number of learners 
earning industry-recognized credentials, and will 
request additional information from the local 
institution if necessary. 


Some states collect data for measures they 
have not clearly defined, particularly industry- 
recognized credentials and work-based learning. 
While 96 percent of states say they collect data 
on industry-recognized credential attainment at 
the secondary level (Figure 2 on page 8), only 
80 percent of states have state-defined lists of 
industry-recognized credentials for second- 

ary institutions. At the postsecondary level, 73 
percent of states say they can measure indus- 
try-recognized credential attainment, but only 


49 percent maintain approved lists (Figure 4). 
How states manage to measure something they 
have not defined is unclear when it is estimated 
that there are more than 300,000 credentials 
in the United States alone.” Additionally, 88 
percent of states say they can measure partic- 


ipation in secondary work-based learning, but 
only 63 percent report that they have a state- 


wide definition for work-based learning at the 
secondary level. One way state leaders can 
improve the accuracy of their data is by pro- 
viding concrete data definitions and business 
rules for measures of career readiness and then 


To the extent practicable, 
states should verify career 
readiness data with other 
agencies, organizations 
and partners to confirm 
that the reported 
information is accurate. 
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providing technical assistance and guidance to help local leaders improve data collection. Without 


setting clear definitions for these measures, states open themselves up for inconsistent reporting 


across the instituti 


A small number o 
organizations and 
conduct technica 


ons and schools they serve. 


assistance and trainings to e 


capacity to enter data correctly. 


This situation does not necessarily mean that s 


their data. Some provided more detail in the su 


which makes comparing approaches across th 


that large numbers of states do not actively vali 


states report validating career readiness measures with data from other agencies, 


partners. Even fewer say they have internal error rules and validation protocols or 


nsure that local leaders have the understanding and 


ates have neglected these approaches to validating 
rvey than others about their validation protocols, 
e states challenging. However, the survey indicates 
date at the state level career readiness measures 


such as work-based learning or industry-recognized credential attainment. As states begin to inte- 


grate measures of career readiness into statewide accountability and reporting systems, they should 


re-examine these practices and ensure that robust processes are in place to validate the accuracy 


and consistency of their data. 


FIGURE 4: DEFINING MEASURES OF CAREER READINESS 


DOES YOUR STATE HAVE A STATEWIDE DEFINITION OF ... 


WORK-BASED LEAR 


NING? DUAL/CONCURRENT ENROLLMENT? 


DOES YOUR STATE HAVE A STATE-DETERMINED LIST OF INDUSTRY-RECOGNIZED CREDENTIALS AT THE... 


SECONDARY LEVEL? 


POSTSECONDARY LEVEL? 
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Many state data systems were constructed to 
support federal accountability and reporting 
requirements rather than a cohesive, statewide 
vision for career readiness. As a result, most 
states have disparate, disjointed data systems 
and multiple collection processes for different 
program areas. This situation creates an addi- 
tional burden for local institutions and service 
providers, which are often asked to respond to 
multiple data collections for the same popula- 
tion of students. Additionally, having separate, 
disconnected data systems makes tracking 
learners as they transition from high school to 
postsecondary education and the workforce 
harder for state policymakers. 


States should instead identify a cohesive, 
statewide vision for career readiness and ensure 
that their data systems allow them to monitor 
learner progress toward that goal. States that 
have prioritized goals around postsecondary 
credential attainment or registered apprentice- 
ships understand that ensuring that learners 
are career ready when they enter, or transition 
within, the workforce is a shared responsibility. 
Tracking progress toward these goals requires 
cross-sector coordination and partnership, and 
states must have aligned and coordinated data 
systems — and use common data definitions — 
to ensure that they can truly achieve a shared 
goal for career readiness. 


Over the past decade, multiple national efforts 
have sought to improve data linkages across 
secondary education, postsecondary education 
and the workforce. These efforts include federal 
investments in state data initiatives through 


SLDS state grants and the Race to the Top Fund, 
which gave states the capacity and resources 
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to create and strengthen data systems.!® ? On 
the workforce side, the federal Wage Record 
Interchange System (soon to be replaced by the 
State Wage Interchange System under WIOA) 
and the U.S. Department of Labor's Workforce 
Data Quality Initiative state grants have provided 
much-needed resources to support workforce 
data collection and alignment, both within and 
across states. 2! To expand access to post- 
secondary data, the National Student Clearing- 
house provides data exchange and research 
services, allowing state leaders to access data 
for graduates who cross state lines.2¢ 


Despite these efforts, only 16 states and the 
District of Columbia have full cross-agency 
data systems that link learner-level data across 
K-12, postsecondary education and the work- 
force.*5 Establishing robust, linked data systems 
that are accessible and actionable takes more 
than the investment of state and federal dol- 
lars; it requires clear definitions and business 
rules, authentic partnerships, full commitment 
and buy-in from agency leaders and elected 
officials and a supportive policy environment. 
While national efforts to improve cross-sector 
data partnership have had some successes, 
states still have much work to do to ensure full 
data alignment across sectors. 


How Aligned Are State CTE Data 
With Other State Data Systems? 


According to State Directors, state CTE data 
systems are mostly discrete and disconnected 
from other statewide data systems, including 
those for K-12 public school students, 
students attending two-year and four-year 
institutions of higher education and individuals 
in the workforce. 
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FIGURE 5: HOW ALIGNED IS YOUR STATE CTE DATA SYSTEM WITH... 


Secondary Data Systems 
18% 


Postecondary Data Systems 


Workforce Data Systems phy 47 
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At the secondary level, many states have student information systems with administrative and 
performance data for every learner enrolled in K-12 public schools. Yet these systems do not 
always include career readiness data. Only 49 percent of State Directors report that their CTE 
data systems are “mostly” or “fully” aligned with their state’s secondary data system. Some states, 
such as Idaho, have successfully integrated their CTE and secondary student record systems 
and adopted uniform statewide course codes so that state CTE leaders do not need to conduct 
a separate data collection to count CTE participants and concentrators. In Idaho, CTE classes 
have six-digit course codes, and all other classes have five-digit course codes, making identify- 
ing learners who are participating or concentrating in a CTE program relatively simple. 


Alignment among data systems Is weaker at the postsecondary level. This situation is not 
entirely surprising, given that many state and national investments in cross-sector data systems 
have focused on the secondary sector. Additionally, state postsecondary institutions are often 
more fragmented than public high schools and school districts. States often have separate data 
systems for two-year institutions, four-year institutions and individual campuses, and these data 
systems rarely include data for private postsecondary institutions. 


According to State Directors, CTE data systems are “mostly” or “fully” aligned with other post- 
secondary data systems in only 28 percent of states. Just as many states report that their CTE 
data systems are “not at all aligned” with postsecondary systems. This lack of alignment is true 
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even for states in which the designated State 
Director is located in the postsecondary sec- 


tor and a high proportion of Perkins funding is 
dispersed at the postsecondary level. 


The lack of alignment is a significant limita- 
tion for state leaders. By definition, CTE spans 
secondary and postsecondary education, 
allowing learners to build upon and apply their 
Knowledge as they transition from high school 
to college. Without strong linkages among 
CTE, secondary and postsecondary data 
systems, state leaders have limited ability to 
assess program quality and evaluate impact. 


In Hawai'i, the University of Hawai'i Commu- 
nity Colleges System partnered with five other 


agencies and organizations to establish the Data 


Exchange Partnership, a full P2OW data system. 
Each of the agencies has set up data sharing 
agreements to share and match learner-level 


data, and Hawaii even used some of its Perkins 
funds to hire a full-time employee to manage 


its state CTE data. The Data Exchange Partner- 
ship allows state leaders to evaluate economic 


outcomes for learners who exit the 


postsecondary sector. 


Even more urgent is the lack of alignment 
between CTE and workforce data systems. Only 
that these 
systems are “mostly” or “fully” aligned, and fully 


18 percent of State Directors report 


37 percent of State Directors say that these 
systems are “not at all aligned.” As a result, 
many states rely on self-reported information 
to determine economic outcomes for CTE 


program participants, limiting the quality 
and reliability of their assessments. 


lowa, however, is one of the few excep- 

tions to this rule. The state has established 
productive working relationships across the 
secondary education sector, the communi- 

ty colleges and the state workforce agency, 
enabling state leaders to access learner-level 
outcomes data for post-program placement, 
employment and wages. One reason the state 
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DATA SOURCES P20W SLDS PORTAL OUTCOME 
K-12 - 
(HIDOE) — —> Ad-hoc Analysis 
UH —> 
> Interactive 
DLIR — Extract, Hawai'i DXP Dashboards 
Transform, and (P20W boaal 
ie > ates —= Interface 
* (Staging Area) Longitudinal 
Other Partners —= Data System) 
_—— Reporting & 
Publishing 
* 
Early Childhood — 
* Data 
Other Agencies — —> Transformation 


Automation 


* In planning stages 


The system design for the Hawai’i Data Exchange Partnership. 
Retrieved from http://hawaiidxp.org/about/system-over view. 


has been so effective at measuring learner 
outcomes is that both the secondary and 
postsecondary sectors use the same unique 
identifiers for individuals in the public educa- 
tion system. Similarly, the lowa Department 


of Education, through its community college 
management information system and its 
relationship with the lowa Workforce Devel- 
opment agency, is able to use Social Security 
numbers as unique identifiers to measure 


education and learner outcomes. 


Additionally, lowa is part of the National 
Student Clearinghouse, which allows states’ 
leaders to track learners who are in further 


education within a state, as well as across 


state lines. The lowa Department of Educa- 


tion also has a long-standing relationship with 


the lowa Workforce Development agency, a 
relationship that has been formalized through 
a memorandum of understanding (MOU). 
The MOU is specifically written for the lowa 


Department of Education to receive wage 
and employment data required under several 
state and federal programs. lowa Is unique 
because its community colleges and sec- 
ondary schools are governed by the same 
department, which has allowed lowa to better 
coordinate data collection across sectors. 
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As long as CTE data systems continue to be isolated and discrete, state leaders will be limited 

in their ability to evaluate learner outcomes and monitor progress toward statewide goals for 
workforce development and education. As states begin the planning process for Perkins V im- 
plementation, they should consider the opportunity to establish new data sharing partnerships 
with other agencies in their states. Through Perkins V, states have a true opportunity to design a 
cross-sector data system that reinforces their state vision for career readiness. The vision should 
serve as a guidepost for collecting, validating and aligning learner-level data across sectors. 
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Separate Data Systems or This work did not happen overnight, though. 
Inconsistent Definitions Before KYStats was established, state leaders 


in Kentucky had been striving for decades to 
One of the consequences of constructing 
; ; ere, write data sharing agreements, gain buy-in 
separate, siloed data systems is that reconfiguring 
from agency heads, and get approval from 
them to connect learner data across program a 
legal counsel. Establishing KYStats took time, 
areas Is harder after the fact. Eighty-eight percent a 
money and dedication. Much 
of states report that separate data systems or : 
of KYStats’ success today can 


inconsistent definitions are a barrier to coordinat- 


a, be attributed to the shared icipati i 
ing, matching and sharing data. This barrier is the a aa Ciba gence 
commitment and trust among 
most common across survey respondents. e Kentucky Department 
agency heads, who recog- aos ; 
Specifically, states say that agencies often hire nized the need for shared © ucation 
different vendors to set up proprietary data data governance and com- e Kentucky Higher Education 
systems at each level, which often leads to mitted themselves to making Assistance Authority 
inconsistencies in data collection. When that vision a reality. : 
; ; ; e Kentucky Education & 
different agencies have their own data plat- : ; 
nr ; Legal or Privacy Barriers Workforce Development 
forms, definitions, business rules and reporting ; 
bie ; Cabinet 
timelines for collecting measures of career Another common barrier 
readiness, having accurate and timely data across the states is legal e Education Professional 
that are comparable across sectors is all the and privacy restrictions on Standards Board 
more challenging. data collection and use. 


e Kentucky Council on 
Seventy-one percent of State 


Establishing cross-sector linkages and clarifying Postsecondary Education 


Directors indicate that these 
data governance takes a lot of time and hard i 
restrictions are a challenge. 
work. In 2012, Kentucky took action to remove 
bts ; States are subject to federal 
alignment barriers by establishing an indepen- 
privacy regulations under the Family Educational 


Rights and Privacy Act (FERPA), but many have 
adopted additional restrictions to protect learner 


dent agency with authority over all education, 


workforce and labor data across the state. 
The Kentucky Center for Statistics (KYStats, 


rivacy. Specifically, states report that restrictions 
formerly the Kentucky Center for Education and : : . ‘ 
ail on accessing Social Security numbers or other 
Workforce Statistics) matches learner-level data ; oe j 

; personally identifiable information make match- 
across sectors, ensures compliance with federal oe 

; ing learner-level data across sectors significantly 
and state regulations, conducts policy research, 


harder. Lengthy procedures for establishing and 
develops reports for state and local policymak- a3. 
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Barriers to Coordinating, Matching and Sharing Data 


States should be commended for taking their data stewardship seriously and prioritizing data 
security and privacy, but they often err on the side of caution at the expense of data quality and 
access. FERPA does not prohibit states from sharing learner-level data across agencies. And even 
when restrictions are in place, states can find innovative solutions by leveraging intermediaries 
such as research institutions or other state agencies to conduct data matching or using alternative 
matching criteria when Social Security numbers are not available. States should be transparent 
about data sharing standards and protocols so that agencies know what they can and cannot do 
with learner-level data. 


Less Common Barriers to Alignment 


Interestingly, aligning data systems appears to be more a function of technology and legal issues 
than political will and resources. Capacity, cost or political barriers are challenges for less than half 


of State Directors, and only one in five states say lack of will from other state agencies is a barrier. 


This news |s good for states. Like Kentucky, states that can harness cross-sector enthusiasm and 
political will can often find innovative solutions to overcome technology and legal barriers. For 
example, when New Jersey found matching student information to workforce outcomes in house 


impossible, state leaders tapped the state Department of Motor Vehicles (DMV), which has access to 
Social Security numbers, names and dates of birth, to facilitate matching secondary and workforce 
data. Through a partnership with the DMV, the Department of Education was able to conduct a 


successful match and access learner-level workforce outcomes for graduates. 


In other states, a unifying state goal or vision for career readiness, such as a postsecondary cre- 
dential attainment goal, has been an effective catalyst for cross-sector partnership around data. In 
2014, Gov. Terry McAuliffe of Virginia issued Executive Order 23, which directed all state agencies 
to work together toward a statewide goal of earning 50,000 credentials in in-demand industries. 
Gov. McAuliffe convened the Governor's Workforce Council, with representation from second- 

ary, postsecondary and workforce development, and all state initiatives since then have coalesced 
around this singular goal. As a result, each of the partnering agencies represented on the Governor's 
Workforce Council are now using common definitions and measures for credential attainment and 


have established agreements to share data through the Virginia Longitudinal Data System. 


FIGURE 6: WHAT BARRIERS PREVENT COORDINATING, MATCHING AND SHARING DATA ACROSS SECTORS? 
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One of the most powerful tools State Directors 


have in their toolbox is their state data system. 
When data systems are well aligned across 
sectors and data are high quality, state leaders 
can make informed decisions to support local 
program delivery, enact effective state policies, 
hold institutions accountable for learner success 
and ensure that all programs are high quality. 


According to State Directors, however, much 


work remains to improve the quality of CTE data 
systems and measures of career readiness. CTE 
data systems are not aligned across the sec- 
ondary, postsecondary and workforce sectors, 
limiting states’ abilities to track learners longitudi- 
nally and evaluate employment and educational 
outcomes. State and federal policymakers have 
started approaching education and workforce 
as part of one ecosystem — working in tandem 
to help learners gain the experience, skills and 
credentials they need for meaningful employ- 
ment — and have enacted policies to encourage 
collaboration and alignment. This approach is 
reflected in common language and measures 
across ESSA, WIOA and Perkins V but must also 
be reflected in state data systems. 


Furthermore, many of the methods states use 


to collect and validate career readiness mea- 
sures are insufficient, leaving states to rely on 
self-reported or unverified data. Many states 
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report that they have limited or no processes to 
validate career readiness data to confirm that 
learners are completing work-based learning, 
earning industry-recognized credentials, earning 
postsecondary credit, or transitioning to 
employment or further education as reported 
by their local institutions. These steps are 
necessary to ensure that data are reliable and 
that career-focused opportunities are delivered 


consistently and with quality across the state. 
States must take measures to improve the 
quality of their data so they can make informed 
decisions based on information they trust. 


Perkins V presents a once-in-a-decade opportu- 
nity for states to reimagine and align their data 
systems — but only if they choose to take it. As 
they prepare to implement Perkins V, states are 
required by statute to engage partners across 
agencies and sectors, collect new performance 
measures, and better connect their data to their 
state and local planning. Perkins V also includes 
new requirements for reporting disaggregated 
data at the state level and using information 
gathered through the comprehensive local 


needs assessment to inform the delivery of CT 
at the local level. The window is wide open for 
states to re-examine their CTE data systems, 
ask intentional questions about data quality, 
and recommit to sharing and using data in an 
impactful way. 
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As states transition their CTE data systems to meet the requirements of Perkins V, they should 
take full advantage of this opportunity to reimagine how their data systems are designed and 
operated. Critical opportunities include: 


Realigning state data systems around a unifying state vision for career 
readiness. State leaders should take a step back and consider whether their data systems can 
sufficiently measure progress toward a shared statewide vision and goals for career readiness. 
They can begin this work by: 


e Taking advantage of Perkins V stakeholder engagement requirements to build relationships 
with state leaders in the secondary, postsecondary and workforce sectors; 


e Building on alignment between Perkins V, WIOA and ESSA to identify common goals, establish 
common measures, and secure buy-in for a shared, unifying vision for career readiness; and 


e Aligning definitions, measures, unique identifiers and collection cycles across program areas 
and, if possible, condensing disparate data systems into one unified system to reduce the data 
collection burden and ensure comparability across programs. 


Auditing data collection processes and identifying opportunities to 
improve data quality. Many states rely on outdated practices and methods for collecting 
and validating career readiness data. They should re-examine existing practices and adopt 
processes that are more robust. States should consider: 


e Moving away from surveys and self-reported data and toward more reliable sources of 
information, such as industry-recognized credential providers, the National Student Clearing- 
house, and administrative data from the secondary, postsecondary and workforce sectors; 


e Embedding robust processes and protocols for validating data at both the local and 
state levels; and 


e Building local capacity to collect and validate high-quality data. 


Committing to transparency and using CTE data to ensure equity across 
all learner populations. Data can be a powerful tool to inform decisions at the state, 
institution and individual levels. States should consider who their key stakeholders are and ensure 
that these stakeholders have access to the data they need. States can begin this work by: 


e Regularly publishing disaggregated career readiness data by sub-population, such as in annual 
report cards, and including all learners, not just CTE students, in reported figures; 


e Building mechanisms to trigger action and support for programs that are not serving each 
learner equitably; and 


e Equipping school leaders to make data-informed decisions so they can ensure that each learner 
can access, fully participate in and succeed in high-quality CTE programs. 
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Taking bold steps to connect data with program approval and funding 
decisions. Many states use their data to inform technical assistance and policy, but fewer 
actually connect their data to decisions about program approval and funding. State leaders 
should take bold actions and use their authority to elevate the rigor and quality of CTE programs 
across the state by: 


e Adopting formal policies that embed data into program application, review and 
funding processes; 


e Coordinating decisions related to evaluating and funding CTE programs; and 


e Building feedback loops to trigger action and support when programs are underperforming. 


A high-quality, integrated data system is necessary to help state leaders identify and close equity 
gaps, continuously improve CTE programs and programs of study, evaluate learner outcomes and 
program impact, and hold institutions accountable for learner success. But there is an urgent need 
to improve data quality and ensure that data systems are aligned across sectors. Perkins V provides 
states the unique opportunity to retool their data systems and secure cross-sector commitment 
toward cohesive, statewide measures of career readiness. 
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